System And Method For Characterizing Particulates in a Fluid Sample

ABSTRACT

A system for characterizing at least one particle from a fluid sample is disclosed. The system includes a filter disposed upstream of an outlet, and a luminaire configured to illuminate the at least one particle at an oblique angle. An imaging device is configured to capture and process images of the illuminated at least one particle as it rests on the filter for characterizing the at least one particle. A system for characterizing at least one particle using bright field illumination is also disclosed. A method for characterizing particulates in a fluid sample using at least one of oblique angle and bright field illumination is also disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/360,832, filed Jul. 11, 2016, and U.S. Provisional PatentApplication No. 62/296,701, filed Feb. 18, 2016, both of which areincorporated herein by reference in their entireties.

BACKGROUND OF THE INVENTION

Protein therapeutics have grown dramatically over the past 25 years andnow comprise 15-30% of the pharmaceutical market. The primary qualityconcern for this class of therapeutics is that they can elicit an immuneresponse from patients who develop anti-drug antibodies. High levels ofanti-drug antibodies can eliminate therapeutic effects by clearing thedrug from the body. This immune response affects 1-10% of patients, whomust stop taking the medication and will return to their initialdiseased state. The presence of particulate matter in these therapeutics(e.g. shed glass from a syringe or a protein aggregate) enhances thisimmune response and the Federal Drug Administration (FDA) thereforeregulates the amount of particles that can be present.

There are existing tools that can provide particle counts and size inthe FDA regulated size range, however there is no instrumentation thatcan routinely and rapidly identify what the particles are actually madeof (see e.g. Bee J S, Goletz T J, Ragheb J A. The future of proteinparticle characterization and understanding its potential to diminishthe immunogenicity of biopharmaceuticals: a shared perspective. J PharmSci. 2012 October; 101(10):3580-5; Ripple D C, Dimitrova M N. Proteinparticles: What we know and what we do not know. Journal ofPharmaceutical Sciences. 2012 October; 101(10):3568-79; and Zölls S,Tantipolphan R, Wiggenhorn M, Winter G, Jiskoot W, Friess W, et al.Particles in therapeutic protein formulations, Part 1: Overview ofanalytical methods. J Pharm Sci. 2012 Mar. 1; 101(3):914-35). There arenumerous analytical instruments used to characterize particulates inprotein therapeutics. They can be classified as particle countingtechniques and particle identification techniques. The main particlecounting techniques for the regulated space of 10 microns and aboveinclude light obscuration (the primary workhorse), membrane microscopy,coulter counters and microflow imaging (MFI). MFI is a newer technologyin this space and has proliferated rapidly. It can provide particlecounts but can be considered a bit of a hybrid because image morphologyand brightness can act as a form of crude particle identification.Specifically, the system can identify oil droplets and air bubbles anddistinguish them from the more solid particles due to their sphericalnature. Further identification is not as trusted but some guesses can bemade by looking at opacity and shape as to whether the particle is aprotein aggregate or a piece of metal. Certainly this identification isqualitative and not definitive. Smaller sized particles in theunregulated space can be analyzed using a variety of instruments whichare not yet quantitative enough in measuring protein therapeutic samplesfor the FDA to regulate this space or make a technique recommendation.These include Nanoparticle Tracking Algorithm (NTA) instruments,Resonant Mass Measurement (RMM) instruments (i.e. the Archimedes system)that can also detect oil droplets, Dynamic Light Scattering (DLS) whichis not quantitative in terms of particle count and the Izon system.

For more definitive identification scientists typically rely on (1)spectroscopy, primarily FTIR and Raman inspection of particles trappedon a filter surface, (2) elemental analysis on an electron microscopeand occasionally (3) fluorescence microscopy after staining particles.Each of these techniques is a powerful and useful way to carry outforensic analysis of particles. In general, they are not routinemeasurement instruments and are only used occasionally due to lowthroughput and the complexity of sample prep. Electron microscopy isexpensive, requires complicated sample prep, highly trained operatorsand high vacuum conditions. Fluorescence microscopy requires staining ofthe particles which is undesirable. In the protein therapeutic space,any changes to the sample, including dilution temperature change,additional reagents added can disrupt the delicate balance of thecarefully formulated sample. There is always a fear that the changes tothe sample will affect the measurement, especially in the case of a dyethat chemically associates with the particles of interest.

Thus, instruments do exist that can identify particles, but they aredifficult to operate and take a long time to provide results, preventingroutine usage. Some materials are more dangerous than others, andknowing what the particles are made of would allow for quickly trackingthe contamination back to its source and eliminating it (see e.g.Rosenberg A S. Effects of protein aggregates: An immunologicperspective. AAPS J. 2006 Aug. 4; 8(3):E501-7; and Carpenter J F,Randolph T W, Jiskoot W, Crommelin D J A, Middaugh C R, Winter G, et al.Overlooking subvisible particles in therapeutic protein products: gapsthat may compromise product quality. J Pharm Sci. 2009 April;98(4):1201-5.) The lack of a routine identification technique means thatscientists typically don't know what's in their samples and cannottherefore detect harmful contamination early enough. Instead, scientistsonly begrudgingly use particle identification equipment duringtroubleshooting efforts due to their tedious operation and lowthroughput. This is a source of great frustration which slows productdevelopment and reduces overall quality and safety.

Current particle analysis systems are also very inefficient. At theformulation selection stage and before manufacturing, researchers havevery little sample available. Current tools requires hundreds ofmicroliters and are very low speed. As a result, researchers often avoidsub visible particle analysis altogether, or conduct such analysissparingly in the earlier stages. It is well known that sub visibleparticle analysis is one of the most sensitive measurements forformulation stability. The ability for researchers to have better andmore efficient analysis tools available at early research stages wouldbe a great improvement to the art.

There is a need in the art for an improved system and method foraccurately characterizing particulates in a fluid sample at highthroughput. There is also a need to conduct sub visible particleanalysis at lower volumes. Overall, there is a need to further improveon the various disadvantages of traditional systems as described herein.

SUMMARY OF THE INVENTION

In one embodiment, a system for characterizing at least one particlefrom a fluid sample includes a filter disposed upstream of an outlet; aluminaire configured to illuminate the at least one particle at anoblique angle; and an imaging device configured to capture and processimages of the illuminated at least one particle as it rests on thefilter for characterizing the at least one particle. In one embodiment,the system includes a luminaire configured to illuminate the at leastone particle at an angle coplanar with a flat plane of the filter. Inone embodiment, the luminaire comprises a plurality of illuminatingdevices. In one embodiment, the plurality of illuminating devices aredisposed radially around and directed inwardly towards the filter. Inone embodiment, the illuminating devices are LEDs. In one embodiment,the system includes a luminaire configured to illuminate the at leastone particle by bright field illumination. In one embodiment, theimaging system is configured to generate a composite image comprisingoblique angle illumination and bright field illumination. In oneembodiment, the imaging system is configured to characterize a pluralityof particles based on a combination of oblique angle illumination andbright field illumination. In one embodiment, a well plate having aplurality of wells that each terminate on a filter. In one embodiment,the filter is a membrane. In one embodiment, the membrane is black. Inone embodiment, the surface has a low surface roughness. In oneembodiment, the well plate comprises a transparent material. In oneembodiment, the well plate comprises a transparent material with anopaque layer on the top surface. In one embodiment, the well platecomprises a reflective film. In one embodiment, the well plate comprisesan opaque bright white material. In one embodiment, the plurality ofwells have a decreasing radius moving towards the filter. In oneembodiment, the system includes a vacuum manifold connected to anegative pressure source and configured for fluid communication with theplurality of wells. In one embodiment, the well plate comprises anoptical feature comprising at least one of a mirror, lens and prism. Inone embodiment, the imaging device is configured to characterize andidentify a material type of the at least one particle using a machinelearning algorithm. In one embodiment, the machine learning algorithmuses observed features including at least one of size, shape, texture,dark-field intensity and intrinsic fluorescence of particles to buildmodels. In one embodiment, the machine learning algorithm is a boostingalgorithm. In one embodiment, the machine learning algorithm uses neuralnetworks. In one embodiment, the machine learning algorithm usesconvolutional neural networks. In one embodiment, the imaging device isconfigured to characterize and identify a material type of the at leastone particle as it rests on the filter using fluorescent imaging. In oneembodiment, fluorescent imaging comprises intrinsic multi channel basedfluorescence. In one embodiment, fluorescent imaging comprises labeledmultichannel based fluorescence. In one embodiment, the imaging deviceis configured to image the filter both before and after the at least oneparticle is captured on the filter. In one embodiment, the before andafter images are processed together using algorithms that processes themto find differences. In one embodiment, the imaging device is configuredto take a plurality of images at a plurality of heights above thefilter. In one embodiment, the plurality of images consists of a highdynamic range set of exposures. In one embodiment, the plurality ofimages are merged together into a single image. In one embodiment, theplurality of images comprises replicates. In one embodiment, theplurality of images at different heights are merged together into asingle image. In one embodiment, the before and after images aremathematically registered. In one embodiment, a median filter is used toprocess the image. In one embodiment, two or more images aremathematically registered. In one embodiment, the imaging device isconfigured to image the at least one particle a plurality of times. Inone embodiment, the imaging device is configured to process the imagesto provide at least one of a number of particles, a size of particlesand a light scattering of particles. In one embodiment, the systemincludes a negative pressure source in fluid communication with openingsin the filter. In one embodiment, the system includes a wicking materialdownstream of the filter and upstream of the outlet. In one embodiment,the filter is a component of a chip. In one embodiment, the chip issubstantially planar.

In one embodiment, a method for characterizing at least one particlefrom a fluid sample comprising: introducing a fluid sample onto afilter; illuminating the at least one particle at an oblique angle; andimaging the illuminated at least one particle as it rests on the filterfor characterizing the at least one particle. In one embodiment, themethod includes the steps of illuminating the at least one particleusing bright field illumination. In one embodiment, the method includesthe steps of generating a composite image based on the oblique angle andbright field illumination. In one embodiment, the method includes thesteps of illuminating the at least one particle by radially surroundingthe at least one particle with a plurality of illuminating devices andilluminating the at least one particle from an oblique angle. In oneembodiment, the method includes the steps of illuminating the at leastone particle in one of a plurality of wells disposed on a well plate. Inone embodiment, the method includes the steps of charactering the atleast one particle based on imaging prior to an after a fluid sample isintroduced onto the filter. In one embodiment, the method includes thesteps of individually and separately imaging each of the plurality ofwells. In one embodiment, the step if imaging comprises imaging at aplurality of heights above the filter. In one embodiment, the pluralityof images comprises a high dynamic range set of exposures. In oneembodiment, the method includes the steps of merging the plurality ofimages together into a single image. In one embodiment, the plurality ofimages comprises replicates. In one embodiment, the method includes thesteps of mathematically registering two or more images. In oneembodiment, the method includes the steps of mathematically registeringbefore and after images. In one embodiment, the method includes thesteps of introducing the fluid sample into a well on a well plate. Inone embodiment, the method includes the steps of illuminating the atleast one particle through an optical feature of the well platecomprising at least one of a mirror, lens and prism. In one embodiment,the method includes the steps of characterizing and identifying amaterial type of the at least one particle using a machine learningalgorithm. In one embodiment, the machine learning algorithm usesobserved features including at least one of size, shape, texture,dark-field intensity and intrinsic fluorescence of particles to buildmodels.

In one embodiment, a system for characterizing at least one particlefrom a fluid sample includes a filter disposed upstream of an outlet; aluminaire configured to illuminate the at least one particle usingbright field illumination; and an imaging device configured to captureand process images of the illuminated at least one particle as it restson the filter for characterizing the at least one particle. In oneembodiment, the system includes a luminaire configured to illuminate theat least one particle at an oblique angle. In one embodiment, the systemincludes a luminaire configured to illuminate the at least one particleat an angle coplanar with a flat plane of the filter. In one embodiment,the luminaire comprises a plurality of illuminating devices. In oneembodiment, the plurality of illuminating devices are disposed radiallyaround and directed inwardly towards the filter. In one embodiment, thefilter is disposed on a chip. In one embodiment, the chip issubstantially planar. In one embodiment, the filter comprises a firstmicropore grid. In one embodiment, the filter comprises a polymer ormetallic membrane.

In one embodiment, a method for characterizing at least one particlefrom a fluid sample includes the steps of introducing a fluid sampleonto a filter; illuminating the at least one particle using bright fieldillumination; and imaging the illuminated at least one particle as itrests on the filter for characterizing the at least one particle. In oneembodiment, the method includes the steps of illuminating the at leastone particle at an oblique angle or an angle coplanar with a flat planeof the filter. In one embodiment, the method includes the steps ofilluminating the at least one particle by radially surrounding the atleast one particle with a plurality of illuminating devices andilluminating the at least one particle from an oblique or coplanarangle. In one embodiment, the method includes the steps of illuminatingthe at least one particle in one of a plurality of wells disposed on awell plate, wherein each of the wells terminates in a filter.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee. The foregoing purposes and features, as well asother purposes and features, will become apparent with reference to thedescription and accompanying figures below, which are included toprovide an understanding of the invention and constitute a part of thespecification, in which like numerals represent like elements, and inwhich:

FIG. 1A is a diagram of a system for characterizing particulates in afluid sample according to one embodiment, FIG. 1B is a diagram of asystem where the fluid sample is pipetted directly onto the filter, andFIG. 1C is a block diagram of a system for characterizing particulatesin a fluid sample according to one embodiment.

FIG. 2A is a diagram of a system for characterizing particulates in afluid sample according to one embodiment where the imaging device hastwo cameras and an imaging region separate from the filter, and FIG. 2Bis a diagram of a system for characterizing particulates in a fluidsample according to one embodiment where the imaging device has onecamera and a single imaging/filter region.

FIGS. 3A and 3B are diagrams of a channel having weir filters accordingto one embodiment.

FIG. 4 is a perspective and partially magnified view of a filter chipaccording to one embodiment.

FIG. 5 is a perspective view of a compact housing that can allow forimaging directly on the filter according to one embodiment.

FIG. 6A is a filter stack within a light ring, and FIG. 6B is anexploded view of the filter stack according to one embodiment.

FIG. 7A is a perspective view diagram of a well plate and membrane, FIG.7B is a photograph of an assembled well plate and membrane according toone embodiment, and FIG. 7C is a perspective view diagram of a wellplate and membrane with an illustration of a fluid sample being insertedinto each well with a pipette.

FIG. 8 is a perspective view of a well plate imaging system according toone embodiment.

FIG. 9A is a cutaway view of a ring light on a well plate, and FIG. 9Bis a photograph of a ring light on a well plate according to oneembodiment. FIG. 9C is a side illumination image and a FIG. 9D is abright filed illumination image that were both used to create acomposite image, as shown in FIG. 9E.

FIG. 10A is a diagram of a well plate constructed as a clearpolycarbonate plate according to one embodiment, FIG. 10B is a diagramof a well plate covered by an opaque material according to oneembodiment, FIG. 10C is a diagram of a well plate that has been paintedwith a reflective coating according to one embodiment and FIG. 10D is adiagram of a well plate made from a bright white polycarbonate accordingto one embodiment.

FIG. 11A is a diagram of a well plate having a neighbor wall designaccording to one embodiment, FIG. 11B is a diagram of a well platehaving a light source inside the neighbor wall according to oneembodiment, FIG. 11C is a diagram of a well plate having a Fresnel lensdesign according to one embodiment, and FIG. 11D is a diagram of a wellplate having a lens design according to one embodiment.

FIG. 12 is a perspective cutaway view of a jig for manufacturing a wellplate according to one embodiment.

FIG. 13A is a diagram of a step of placing a consumable onto a vacuummanifold having a wicking material according to one embodiment, and FIG.13B is a diagram of a step of pulling a sample through the membraneaccording to one embodiment.

FIG. 14A is a flow chart of a method of characterizing a particle usingoblique angle illumination according to one embodiment, and FIG. 14B isa flow chart of a method of characterizing a particle using bright fieldillumination according to one embodiment.

FIG. 15A is a gallery of tracked particles according to one embodiment,FIG. 15B is a MFI particle tracking graph according to one embodiment,and FIG. 15C is a histogram of particle size according to oneembodiment.

FIG. 16A is a diagram of a system where samples were added to a well andpulled through the mesh using vacuum pressure regulated by a pressurecontroller according to one embodiment, and FIG. 16B is an image of amesh following separation of a mixture of aggregated lysozyme (2-50 um),stainless steel beads (10-20 um), and latex beads (15 um) according toone embodiment.

FIG. 17A shows Raman data and outlines that were added to showtopographical similarities according to one embodiment, and FIG. 17Bshows Rapid FTIR imaging data of a sample with a heat map of the Amid Ipeak at 1700 wavenumber which identifies proteinaceous materialaccording to one embodiment.

FIG. 18A shows a “before” image where and 5 particles were identified inthis section of the before image according to one embodiment, FIG. 18Bshows an “after” image identifying a good “after” threshold as describedin detail below where the “after” image shown identified 16 particlesaccording to one embodiment, and FIG. 18C shows the 5 particles from the“before” image connected to the “after” image, netting 11 new particlesthat will be reported in this section of the membrane.

FIG. 19A depicts particles on a hex filter showing differentiationbetween two types of materials via fluorescence characterizationaccording to one embodiment, and FIG. 19B depicts the hex filter'sability to distinguish between ETFE polymeric standards that mimicprotein aggregates with actual protein aggregates.

FIG. 20A shows a schematic for building a machine learning modelaccording to one embodiment, and FIG. 20B shows an example output plotof protein vs non-protein identification according to one embodiment.

FIG. 21A is an out of focus image of particles and FIG. 21B is acorrected image of particles after a focus fusion technique is applied.

DETAILED DESCRIPTION OF THE INVENTION

It is to be understood that the figures and descriptions of the presentinvention have been simplified to illustrate elements that are relevantfor a more clear comprehension of the present invention, whileeliminating, for the purpose of clarity, many other elements found insystems and methods of characterizing particulates in a fluid sample.Those of ordinary skill in the art may recognize that other elementsand/or steps are desirable and/or required in implementing the presentinvention. However, because such elements and steps are well known inthe art, and because they do not facilitate a better understanding ofthe present invention, a discussion of such elements and steps is notprovided herein. The disclosure herein is directed to all suchvariations and modifications to such elements and methods known to thoseskilled in the art.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although any methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of the present invention, the preferred methodsand materials are described.

As used herein, each of the following terms has the meaning associatedwith it in this section.

The articles “a” and “an” are used herein to refer to one or to morethan one (i.e., to at least one) of the grammatical object of thearticle. By way of example, “an element” means one element or more thanone element.

“About” as used herein when referring to a measurable value such as anamount, a temporal duration, and the like, is meant to encompassvariations of ±20%, ±10%, ±5%, ±1%, and ±0.1% from the specified value,as such variations are appropriate.

Ranges: throughout this disclosure, various aspects of the invention canbe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Where appropriate, the description of a range should beconsidered to have specifically disclosed all the possible subranges aswell as individual numerical values within that range. For example,description of a range such as from 1 to 6 should be considered to havespecifically disclosed subranges such as from 1 to 3, from 1 to 4, from1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well asindividual numbers within that range, for example, 1, 2, 2.7, 3, 4, 5,5.3, and 6. This applies regardless of the breadth of the range.

Referring now in detail to the drawings, in which like referencenumerals indicate like parts or elements throughout the several views,in various embodiments, presented herein is a system and method forcharacterizing particulates in a fluid sample.

With reference now to FIG. 1A, a system 100 for characterizingparticulates 130 in a fluid sample is shown. The system 100 includes achip 110 having a fluid channel 112 that is configured to allow a fluidsample to flow downstream through an imaging region 114 and a filter118, before exiting through an outlet 122 (underneath the filter in theview of FIG. 1A). In one embodiment, the outlet is any space or void inthe system downstream of the filter that is in fluid communication withopenings in the filter. The downstream direction is relative to thefilter. Thus, downstream is the direction that a fluid sample wouldnormally move as it flows towards and/or through the filter (e.g.laterally east or west and/or down in certain embodiments). In certainembodiments, the flow is generated by a negative pressure sourceapplying a vacuum pressure to the fluid channel or the outlet. Incertain embodiments, the outlet 122 is underneath or otherwisedownstream of the filter 118. The filter 118 has multiple pores 119through which fluid can flow, while trapping particles 130 presentwithin the fluid sample. Advantageously, embodiments of the systemprovide high throughput particle identification that is at least anorder of magnitude faster than conventional systems, while particlecounting information is consistent with state-of-the artinstrumentation. Particles 130 are imaged by an imaging device 140 asthey travel down the fluid channel 112 and are caught at the outlet 122by a filter 118, which in certain embodiments is a high densitymicropore grid which acts like a sieve. The sieve with capturedparticles resting on top can be imaged, for example by FTIR to provideparticle identity. Whereas this type of analysis using conventionaltechnology can take several hours to accomplish, embodiments describedherein are able to carry out both counting and typing within 30 minutesor less. The additional typing capability of a routine particle countinginstrument accelerates clinical development and enables higher quality,safer therapeutics.

In one embodiment, fluid samples are injected into a chip 110 having afluid channel 112 that includes an imaging region 114 and a filter 118.Particles 130 are imaged as they travel down the fluid channel 112, 150.In one embodiment, the particle analysis is microflow imaging (MFI),which is currently a preferred method of routine characterization in thecompendial (regulated) particle range. In conventional MFI systems, thefluid sample is imaged then pumped to waste. In embodiments describedherein, the outlet 122 of the channel 112 is blocked by a filter 118,which in certain embodiments is a grid of lithographically defined pores119 which act like a sieve. Particles 130 that are larger than the pores119 are trapped by the sieve. The grid is very dense and can fit withinthe field of view of the camera. Particles 130 are imaged multiple timesand tracked to their landing site in the filter 118, 152. After thesample has been fully pumped through, imaging (e.g. FTIR imaging) isused to characterize and identify the trapped particles 130, 154 andassign a material type. The nicely defined grid on the filter 118 isideal for rapid scanning and spectra for each particle can be linked tothe particle's image (from step 150). In certain embodiments, dynamicfluid delivery and control systems are implemented. As the filter isfilled up, the flow resistance will increase, and the presence of flowsensors and pressure feedback can dynamically change or maintain theflow rate. Otherwise, poor control could either rupture the membrane orpush deformable particles through the membrane.

In one embodiment, the channel 112 is imaged using magnification opticsand with a bright blue LED flash that is timed with a camera exposure.This method of delivering particles into a flow cell and imaging them ishow current MFI instruments work (e.g. ProteinSimple, Fluid ImagingTechnologies). For MFI analysis, careful automation can be used to timethe flash illumination and exposure together with the fluid flow so thatparticles are not missed between exposures. In certain embodiments,numerous pictures of each particle are taken in order to track them andcorrelate the images to FTIR analysis.

In certain embodiments, the chip has a shorter fluid channel or no fluidchannel at all, and fluid samples are pipetted directly onto the filter(see for example FIG. 1B). In this type of embodiment, imaging andparticle characterization can be based solely on imaging particles onthe filter. In certain embodiments, filters may be stacked in series sothat the holes get progressively smaller. For example, one embodimentcan include 3 filters (a 25 um filter, 10 um filter and a 2 um filter).In certain embodiments, filter material can be modified to suit the formof spectroscopy used. In the case of FTIR, a transparent membrane can beused (for example silicon nitrode or silicon dioxide). In the case ofRaman spectroscopy, a metallic coating of gold or silver can bedeposited onto the surface. In certain embodiments, the filter surfacecan be modified with anti-fouling agents (e.g. PEG, functional silanes,surfactants) to avoid fouling or reduce flow resistance or capillarypressure of pores (e.g. with hydrophilic PEG coating). In certainembodiments, pore shape in all three dimensions can be preciselycontrolled. For example, conical pores can be utilized to improvecapture efficiency and trapped location accuracy of filtered objects. Incertain embodiments, consistent pore size and density is utilized tolead to consistent transmission of light. Reduction in lighttransmission can also be used to quantify trapped particle concentrationand/or determine composition considering wavelength of excitation.

In one embodiment, particles larger than the pore size will be trapped.In one embodiment, since both the main channel and the grid will bewithin the field of view of the camera, particles can be trackeddirectly to their landing site on the pore grid. The pore grid can bemade by optical lithography which can precisely pattern tightly packedholes with a high amount of total open area. In one embodiment, eachparticle, after it is tracked to its landing site will have a highquality image associated with it (from step 150) (just as in the MFIsystems) and an x and y grid position specifying its location. This isimportant for the next step where the identification is performed. Oneadvantage of tying images to spectra is for additional corroboration,e.g. particles that look like protein aggregates and also have a proteinaggregate spectrum provide added confidence. In certain embodiments,multiple images of the same particle are used to provide a higher levelof morphological and/or optical scattering intensity analysis.

In certain embodiments, in order to make particle tracking moreaccurate, a more sophotiscated particle location prediction is utilized.Imaging particles in the flow stream with the filter in the field ofview makes it possible to connect the landed particle to the flowingparticle and tie together spectroscopy and morphology. Predictionmethods are based on hydrodynamics and improve the accuracy of thespectroscopy-morphology linking. In certain embodiments, the followingpieces of information can be utilized to make a good prediction: (1)Particles generally stay within their streamline. If flow is from leftto right, particles flowing at one “latitude” will likely land at aroundthe same latitude. (2) Landing particles will be seen on the filteritself. (3) A video of particles can be used to identify particlevelocity. There will be low Reynolds number Poiseuille flow conditions.Particle velocity allows the prediction of height of the particle withinthe channel and therefore allows prediction of a possible landing site.For example, particles with the fastest velocity will be in the middle(height-wise) of the channel. If flow occurs from left to right,particles will likely land in the center “longitudes” (i.e. East-West)of the filter.

In certain embodiments, particle imaging occurs after the particles havelanded on the filter (rather than in the flow cell as is explained abovein step 150 and 152). Multiple images can be taken throughout thefiltration process so that particles which land close to each other canbe distinguished by observing their landing times. The grid can betransparent allowing for many different types of imaging. Since theparticles are stationary after they have been captured, long exposuretimes and multiple imaging methods can be used sequentially of the sameparticles and will allow images to be overlaid and so that each particlecan be linked between images.

Rapid particle identification (e.g. FTIR imaging, step 154) takes placeafter all of the sample is run through the channel 112 and particles 130are collected on the filter 118. FTIR imaging is used to carry out highthroughput spectroscopy on the tightly packed and neatly arrangedparticles. FTIR spectra can be used to identify the particles but alsocarry out protein structural analysis on protein aggregates to learn theextent of the damage[38]. FTIR is a form of vibrational spectroscopywidely used in protein therapeutics. There are at least two advantagesfor selecting FTIR imaging over Raman spectroscopy, which is anotherpossible choice in alternate embodiments. First, time savings—sincethese particles are packed neatly into a tight grid, FTIR imaging can beused to analyze numerous particles at once unlike Raman which has sloweracquisition times and must scan a laser spot. A Raman scan that mighttake several hours can be compressed down to 5 minutes with an FTIRimaging system. This is critical to make the system a routine analysissystem. Second, it is preferred by segments of the market since FTIR isfaster. Although neither Raman nor FTIR is used for typing metals, thesecan often be categorized (if not explicitly typed) by their highlyopaque nature. Certain embodiments include removal of the filter grid(or transfer without removal) to allow the user to transfer it to asystem more suitable for metals—e.g. laser induced breakdownspectroscopy (LIBS) or SEM EDS analysis. In certain embodiments, acustom library of materials can be built by using a database of spectraor by manually measuring materials that are possible particle producers(such as pump parts, syringe stoppers, glass vials, excipient, siliconeoil, the pharmaceutical, etc.) and storing their spectra. Preselecting alibrary of materials will dramatically increase the accuracy ofspectroscopy results and vastly improve user experience and convenience.

Advantageously, a commercial chip according to embodiments describedherein are manufacturable, scalable (e.g. $5 cost or less at volume) andcan capture the size range of interest (>0.5 μm). In certainembodiments, throughput is 30 minutes per sample with >90% of particlestracked and identified. In certain embodiments, software is used toautomate system operation and data processing. In certain embodiments,there is >90%, >95%, >99% or 100% agreement with conventional particlecounting instruments.

With reference now to FIG. 1C, a block diagram of a system 160 forcharacterizing particulates in a fluid sample is shown. The systemincludes an imaging device 180 which can include one or more cameras168, a light 170 for illuminating fluid samples and particulates (suchas the ring light described herein), a controller 162 for controllingthe camera 168 and light 170, and a memory module that communicates withthe controller 162. The controller 162 can process images as describedin the various embodiment. As understood by those having ordinary skillin the art, the controller 162, memory 164, camera 168 and light 170 cancommunicate via a number of configurations, and communication can bewired or wireless between system components. Generally, the imagingdevice 180 in certain embodiments includes the camera 168 and acontroller 162 in any configuration for communication with one another,such that the imaging device 180 can capture and process images, andoutput results of particulate characterization for the user, such as toa display 166. The controller 162 can be integrated into the camera 168,located elsewhere in the system 160 or located remotely and otherwisecommunicating with the camera 168, the light 170 and other system 160components. The controller can connect to a display 166 forcommunicating results and images to the user. The display 166 can betouch screen for providing user input into the system 160. Thecontroller 162 can also communicate with a stage 172 for which the fluidsample or chip is placed on. In certain embodiments, the stage moves tocenter the sample under the light and camera. The stage can also haveillumination elements controlled by the controller.

With reference to FIG. 2A, the imaging device 140 can include a firstand second camera 142, 144 for imaging the imaging region 114 and thefilter 118 respectively. In one embodiment, a single camera is used toimage both regions. In certain embodiments, more than one camera is usedto image each respective region. In one embodiment, a single camera isused to image the filter and fluid samples are provided directly ontothe filter. As shown specifically in FIG. 2B, the imaging device 140′can use a single camera 142 for embodiments where the imaging region andthe filter are the same region 118, such as when the system is onlyconcerned with imaging particles trapped on the filter. This can be thecase for example when fluid samples are pipetted directly onto a filter(e.g. see FIG. 1B). It will be understood by those having ordinary stillin the art that the imaging device can process images for characterizingparticulates via an integrated controller or one or more separatecontrollers communicating with the imaging device and the system. Memorymodules for storing software and images, input/output components andother components normally found in similar types of systems are alsopresent and can be configured in any way as would be apparent to thosehaving ordinary skill in the art. In certain embodiments, quantitativehigh fidelity particle imaging can be accomplished directly on thefilter rather than during flow. In certain embodiments, observation witha microscope allows for particle imaging for size and morphologycharacterization prior to immobilization on filter, which would allowhigher confidence size measurements and potentially 3D reconstructionsof particle morphology.

In certain embodiments, alternate filter types are used. With referencenow to FIGS. 3A and 3B, in one embodiment, a weir filter 200 can also beused to capture particles instead of the microporous grid. These filters200 are in-line with the channel and can be used to pack particles intoa much denser pack, allowing for higher throughput spectroscopy (seeFIG. 3B). In one embodiment, there are multiple weir filters in seriesthat get progressively smaller the further downstream. The shape of theweir filter can vary. In one example, to increase surface area of thegap, rather than extending the weir straight across the channel in aline, the weir is formed into a different shape, for example a ‘V’, ‘U’or serpentine. In one embodiment, a crude estimate of particle counts ormass can be made by measuring the area of the packed mass of particlesand combining this with the weir type and channel depth.

In certain embodiments, particle size-based sorting can be used upstreamso that particles of a certain size land on certain parts of the filteror can pass through the appropriate weirs more easily. For example, aparticle sorter can arrange it so the larger particles are positionedtowards the left with respect to the direction of flow and smallerparticles move towards the right. In this way, when particles land onthe grid filter they can be easily quantified. In the case of the weirfilter, large particles and small particles, when so sorted, will not bein the same fluid stream line. This prevents a circumstance where thelarger particles get captured in a weir and obstruct the smallerparticles which ideally would escape the large particle wier and getcaptured further downstream. In certain embodiments, micromesh allowsrepeatable flow-rate vs pressure dependence through low variability inflow resistance across devices, improving run to run consistency.

In certain embodiments, the system is made from inorganic materials toallow for harsh chemical treatment/cleaning (e.g. with Piranha/sulfuricacid/hydrogen peroxide, bleach) for subsequent re-use. In certainembodiments, wafer grade materials can be chosen such that extremeflatness of device permits large area scanning without the need torefocus optics for imaging, or excitation/collection of spectroscopicsignals. In certain embodiments, micropatterned fiduciary marks can bepatterned onto surface for automated imaging steps. For example, afiducial mark can indicate where image or spectroscopy scanning begins.Another example is that multiple marks in different locations acrossdevice can be imaged for the purpose of quantifying device tilt and bow.In certain embodiments, micropatterning of devices allows for additionof alignment features (e.g. holes for guide pins) etched right into thedevice material for precise alignment in assemblies (e.g. flow assembly,imaging assembly) or stacking devices in series.

Thus, compared to conventional devices, embodiments described hereinhave several advantages. With respect to particle identification, byimaging the particles as they land on the filter, each particle'sspectra can be directly tied to its image, giving it a precise identity.This feature is highly desirable to scientists. There are also mayfiltration benefits. Lithographically defined pores have excellentqualities. They have a large amount of open area which greatly increasesvolumetric flow rate while reducing the pressure required compared totraditional filters. The result is a much gentler filtration and reducedchance of flexible particles being pushed through the filter. In certainembodiments they are made of materials that are ideal for spectroscopyand the nature of the tightly packed grid dramatically shortens thespectroscopic analysis time. Embodiments described herein overcomeshortcomings of standard filtration. Traditional filtration microscopyis perceived as less quantitative than microflow imaging becauseparticles can be pushed through the filter or difficult to image once onthe solid substrate. However, ty combining the two techniques, thequantitative counting of MFI can be leveraged while taking advantage ofthe typing capabilities of filter spectroscopy. Regarding flow control,as particles accumulate on standard filters, the pressure can build upand force deformable particles through the filter. The chip according toembodiments described herein allows dynamic control of fluid pressure onthe membrane (e.g. by reducing flow rate) to keep the stress on eachcaptured particle below a dangerous threshold. Flow control can alsodetect filter blockage. High throughput spectroscopy is anotherimportant advantage. Traditional IR and Raman microscopy requiresscanning a point source and single element detector to generate acompositional map. Arrayed detectors used in IR imaging offersignificantly higher throughput through parallel acquisitions ofspectra. In addition, sample banking is improved according to thevarious embodiments: The use of disposable chips allows each sample tobe banked should the need arise for future, deeper investigation orbackup to FDA QC record keeping. Chips can also be “opened” and analyzedwith electron microscopy or LIBS.

With reference now to FIG. 4, in one embodiment, to fabricate a filterchip 300, standard microfabrication techniques conventionally used forcell separation devices are utilized (see e.g. Earhart C M, Wilson R J,White R L, Pourmand, N, Wang S X. Microfabricated magnetic sifter forhigh-throughput and high-gradient magnetic separation J. Mag. Mag. Mat.2009 May 1; 321(10):1436-39; and Earhart C M, Hughes C E, Gaster R S,Ooi C C, Wilson R J, Zhou L Y, et al. Isolation and mutational analysisof circulating tumor cells from lung cancer patients with magneticsifters and biochips. Lab Chip. 2014 Jan. 7; 14(1):78-88.). In oneembodiment, a double polished silicon wafer 302 is coated with a 3micron thick layer of silicon dioxide 304, in which micro-pores 306 willbe patterned by optical lithography. These pores 306 will be opened forfluid flow by etching a honey-comb structure through the backside,terminating at the oxide layer, by deep Reactive Ion Etching. Largerholes for fluidic and illumination access can also be etched in thisstep. In one embodiment, the filters are 1×2 cm rectangular dies with apatterned filter area of 2×4 mm, containing 240 hexagonal arrays with300-1500 pores per array, yielding 72,000-360,000 pores per devicedepending on pore size (2-5 um) and spacing. In one embodiment, thefilter capacity is determined sufficiently high such that if a samplecontains the particle concentration limit stated in the PharmacopeialConvention (6000 particles/sample), the fluid resistance will change byless than 10%, assuming 100% of particles are trapped and one particleoccupies each pore. Adjacent to the patterned pore area, separated by1.5 mm, is a 2×4 mm oxide window 308, through which illumination will beprovided for microflow imaging. Both regions have sized to fit in thefield of view provided by using a 4× objective and a camera with sensorsize of 22.5×16.9 mm. One issue addressed by this design is to ensurethat the illumination membrane will be mechanically stable. As a riskmitigation step, according to one embodiment, a design with a honeycombsupport structure is implemented. Although it may prove more difficultto image particles while directly over the support structure, the highsampling frequency should enable imaging of a particle while over theoxide membrane as it traverses the imaging region. As shown in FIG. 5, acompact housing can allow for imaging directly on the filter. In oneembodiment, an inlet can 408 and outlet 410 are in fluid communicationand the outlet 410 is covered by a filter 404. The filter 404 issurrounded by an o-ring 406 and covered by a cover slip 402.Accordingly, the filter 404 can be imaged directly for particlemorphology as it is built into a simple housing.

With reference now to FIGS. 6A and 6B, a method for illumination isshown according to one embodiment. A filter stack 502 includes an opaquetop ring 504, and transparent or translucent double sided tape 506, amicrofilter chip 508, double sided tape 510 and a bottom ring 512. Theassembled filter stack 502 is configured to fit within the ring lightassembly 500, which includes multiple lights (e.g. LEDs) pointed inwardstowards the chip 508. The quality of particle imaging on the microfilterchip 508 is dramatically affected by the method of illumination. Anillumination method where light is directed towards the particlein-plane with the chip surface provides excellent illumination of theparticles while preventing excessive illumination of the pores.Illumination of the pores can cause difficulty when processing theimages since they can be confused with particles. To achieve thiseffective illumination, a ring light 500 of LEDs 520 was developed wherethe LEDs 520 point inward radially. The ring light 500 is positionedsuch that the LEDs 520 are in-plane with the top of the microfilter chip508. This method is similar to a dark field illumination whereillumination comes from the side and object edges are appear bright.However true dark field illumination is not in-plane with the image. Inthe case of the microfilter chip 508, the pore edges are alsoilluminated in dark field which is not desired.

In most filter applications, the filter is placed in a holder that willblock or distort illumination that comes from the side. A filter holderhas been developed that uses a transparent or translucent tape layerthat allows light to travel through this layer to illuminate the samplein an effective way. The construction of the consumable is as follows. Aring-shaped double sided tape adhesive 506 is placed on top of themicrofilter chip 508. The tape adhesive 506 has a gap in the center toallow the liquid to pass through the filter part of the chip 508. Aring-shaped top part 504, currently made of opaque acrylic is placed ontop of the double sided tape 506 so that the tape 506 is sandwichedbetween the chip 508 and the top acrylic ring 504. This forms areservoir on top of the microfilter chip 508 which is fluid tight.Liquid sample can be dispensed into the reservoir and vacuumed through.After vacuuming, the top surface of the microfilter chip 508 can beilluminated through the tape layer. An opaque top ring was found to besuperior to a clear or translucent ring. This is because confining lightinto the tape layer ensures more planar light reaches the sample.

In one embodiment, high accuracy classification of particles imaged on afilter can be achieved by applying machine learning algorithms to datacollected on the particles. Data can include images, spectroscopic orfluorescence signals or spectra, or features extracted from images,spectroscopic or fluorescent signals or spectra through imageprocessing, signal processing, or an unsupervised learning platform. Thedata can be analyzed by a single or combination of machine learningalgorithms, for example, random forest, boosting, or artificial neuralnetworks, to generate a predictive model. A training set of datacollected on particles of known identity is used to build the predictivemodel, which can then be applied to unknown particles for typing orclassification. Classification information can include particle type(e.g. protein, glass, hybrid materials) or sub-types (type of glass,metal, or protein). Classification can also include aggregation state orlikely cause of aggregation for protein particles. In one embodiment,the machine learning algorithm is a boosting algorithm. In oneembodiment the machine learning algorithm uses neural networks. In oneembodiment the machine learning algorithm uses convolutional neuralnetworks. Machine learning algorithms can also be applied to data toproduce a continuous output instead of or in addition to classification;for example, degree of protein aggregation, denaturation, orcrystallization for protein aggregates.

In one embodiment, high vacuum pressure (i.e. low chamber pressure belowthe chip) has the advantage of driving fluid quickly through the filteror membrane, reducing overall processing time. However, high vacuumpressure also runs the danger of driving delicate particles through themembrane that would be desirable to capture instead. In order to drivefluid through the membrane, a breakthrough pressure must be achieved.The breakthrough pressure is often quite high in order to overcomecapillary forces that may develop at the filter pores. Once flow isestablished, a different vacuum pressure, often lower in strength, canbe applied. In order to reduce this breakthrough pressure and avoid thepossibility of driving particles through the filter pores, a variety ofthin surface treatments may be employed to increase the hydrophilicityof the filter surfaces. Examples of these treatments are the use ofbovine serum albumin, hydrophilic silanes that can be vapor deposited,hydrophilic thiols that can self assemble on gold surfaces of themembrane, and poloxamer pluronic F-68.

In one embodiment, precise characterization of particles using texture,fluorescent intensity, and morphology requires accurate measurement ofthe intensity of each particle. Traditional imaging of particles is doneby taking a single image and minimizing the number of saturated pixels,this is known as a standard-dynamic-range imaging (SDR). When workingwith particle populations that have order of magnitude size differencesthe information collected using SDR imaging requires that the intensitydata from small particles is very low and may even be indistinguishablefrom the background of the image. High-Dynamic-Range imaging (HDR) usescomputer algorithms to merge many SDR images taken at exposure timesranging many orders of magnitude into a single high bit depth (16+)Image that contains the full dynamic range of the image. Utilization ofHDR imaging results in Images with a significantly larger range ofluminance levels than can be achieved using traditional methods andallows for better characterization of particles.

In one embodiment, each particle analyzed will contain a uniqueintensity map of the particle that can describe the surface of theparticle. The surface properties of a particle are unique to theparticle type and in some cases sub-type of the particle. These surfaceproperties can be used in conjunction with other properties to classifyparticles. Information about the surface of the particle can beextracted using mathematical algorithms such as the gray-levelco-occurrence matrix, local binary partition, and edge density anddirection.

In one embodiment, droplets of liquid that adhere to the back of thefilter (the non-imaging side) can disrupt the imaging by filling pores,entering back into the top of the chip and obscuring particles andparticle edges. Several methods have been employed to remove this liquidfrom the chip. In one embodiment, the method is to vacuum the chip for asufficient time using a vacuum source capable of high flow rate (atleast 2 cubic feet per minute). In one embodiment, the method is to aima flow of gas (preferably dry, like nitrogen gas) towards the top of thechip. In one embodiment, the method is to force pressurized gas throughthe surface rather than just aiming a flow of gas towards the surface asin the second method. This requires a seal between the gas source andthe chip. In one embodiment, the method is to use a fan above or belowthe chip to drive air through the membrane. Using a fan below thesurface is preferable since particles will be driven against themembrane and there is less chance of particles being blown off as in thecase of the fan above the chip. In one embodiment, the method is to usea wicking material (such as a cellulose pad or glass fiber pad) andapplying it to the bottom of the chip so that the liquid enters thewicking material.

With reference now to FIGS. 7A and 7B, in one embodiment, a disposablewell plate 600 includes multiple fluid channels or wells 606 for fluidsamples. The well plate can include multiple wells, including 32-96wells, up to 384 wells, up to 1536 wells or more. In one embodiment, thedisposable well plate 600 consists of two layers 602, 604. A top layer602 which is a polycarbonate (transparent) plate with holes 606, and abottom porous membrane material layer 604 which is thermally bonded tothe polycarbonate 602. The pore size of the porous membrane (alsopolycarbonate) is currently 400 nm and is commercially available. In theone embodiment and as shown in FIGS. 7A and 7B, two discs, each 47 mm indiameter are used. Two discs are assembled on the plate 600 to form 32sample wells 606. Membranes are bonded to the plate using a thermalbonding technique. In one embodiment, the wells form a circular patternto fit within the ring light described herein. The fluid sample can beintroduced into each well by pipette, as illustrated in FIG. 7C

A robotic apparatus for imaging the well plate is shown with referencenow to FIG. 8, according to one embodiment. The well plate 600 sits on amicroscope stage 702 that further sits on a track system 704 that canmanipulate which well or set of wells is positioned under the ring light706. Imaging and processing components 710 are configured above the deckfor imaging the fluid samples and particulates in the wells. Generally,the instrument is a robotic microscope with a camera and illuminationmethod as described herein. In one embodiment, a 32 well plate 600 isplaced on the microscope stage 702 and the microscope automaticallypositions wells under the objective lens for imaging. Large area (e.g.4/3 inch) camera sensors can be utilized for imaging, which allows goodimages with low magnification lenses. This helps fit a large area of themembrane in a single image, a configuration that is designed for speedwithout undue sacrifice of imaging performance.

In one embodiment, well plates are loaded into the instrument 700 andimages are taken of the plate to generate at least one “background”image. In certain embodiments, multiple background images are actuallytaken. The background image can be generated in certain embodimentsbased on multiple averages, multiple exposure settings, multipleillumination methods, multiple focus heights, and comparable techniquesknown in the art. There are likely some particles or deformities on themembrane present before the introduction of the sample that should beaccounted for and in general, there are always background features onthe membranes that must be accounted for. These can be scratches,texture, particles from manufacturing or from the air, etc. In order toget an accurate particle count from a sample, these background featuresshould be taken into consideration. A “before image” is acquired forthis purpose and analyzed before samples are loaded onto the membraneand vacuumed. The “before” and “after” images are processed using avariety of algorithms to get accurate particle data.

Next, the user can load plates with fluid by pipetting samples into thewells. Then, plates are placed on a vacuum manifold and vacuum isapplied so that only particles are left behind on the membranes (nofluid). Next, the user can load plate into instrument for imaging andanalysis. The instrument then images each well under 1 or more opticalconditions (eg. bright field, darkfield, fluorescence, differentexposures). In certain embodiments, a composite image is formed using acombination of bright field and oblique angle illumination. Combiningbright field with side illumination yields excellent differentiationbetween particles of different materials. In certain embodiments,through-plate oblique illumination is used. Next, the instrument movesto the well, focuses using a software technique, and acquires imagesusing one or more techniques. For each technique a multi exposure stackof images can be taken. These stacks can then combined into a singlehigh dynamic range image. If the whole membrane doesn't fit on oneexposure, multiple areas are imaged separately as described above andcombined into a single stitched image. In certain embodiments, a focusfusion and averaging technique is utilized. The HDR image stack can betaken at many different heights and focus fused together. Also, everyimage can really be an averaged image consisting of 1 or more repeats atthe same exposure. In certain embodiments, to do the focus fusion, eachimage needs to be registered to each other to account for slightmovement between images. In the next step, the system then proceeds ontothe next well. Images are analyzed using an image processing steps toisolate particles and identify which particles came from the sample andwhich were background particles from the before image. In certainembodiments, both images undergo heavy image processing and the beforeimage is combined mathematically with the after image in such a way thatthe background texture and particles are eliminated. The instrument thenoutputs desired physical and chemical parameters, such as size+counts,morphological properties (e.g. aspect ratio) particle ID, brightness,opacity, a ratio of brightness between two different imaging modes, orother stability parameters. Next, the user can take out the plate andrun additional chemistry for a new assay that is brought back to theinstrument for conducting differential measurement (stability,solubility, activity assay). After particles are collected, they can befurther analyzed by additional techniques (other assay, FTIR, EDX), etc.Advantageously, the plates capture the particles and present them in anice way to these other instruments.

As was the case with previous embodiments, a key to instrument advantageis the method of illuminating the particles on the membrane, whichyields high signal to noise ratios. The objective is to illuminate thesamples in such a way that the particles appear as bright as possiblecompared to the membrane, which forms the background of the images. Thecomponents involved in this method include an LED ring light, theconsumable plate, and the membrane. The illumination source is an LEDring light, which illuminates particles at a very oblique angle, asshown in FIGS. 9A and 9B. The LED ring 804 uses surface mounted LEDs,which direct the light towards the center of the ring 804. The LEDs areplaced as close to the membrane as possible so as to achieve the mostoblique light possible. It is believed that this oblique angle causesthe light to preferentially scatter off of the particles compared to themembrane surface. This results in increasing the particle signal tonoise ratio. In some embodiments, the oblique angle is less than 17degrees. In some embodiments, the oblique angle is less than 13 degrees.In certain embodiments, the system is configures to that the angle iscoplanar with the surface. In one embodiment, the oblique angle is anynon 90 degree angle relative to a flat plane of the filter. In oneembodiment, the oblique angle is 30 degrees or less. In one embodiment,the oblique angle is 20 degrees or less. In one embodiment, the obliqueangle is 17 degrees or less. In one embodiment, the oblique angle is 13degrees or less.

In one embodiment, the light coming from the LEDs will interact with theplate before reaching the particles. In one embodiment, a transparentplate is utilized with polished walls that allow the light to enter andexit the plate with little scattering. In one embodiment, the plate iscoated in a reflective film. Although the light cannot enter the bulkmaterial of the plate in this case, the image quality is still high.This is likely due to the light entering a well and reflecting insidethe well while maintaining the oblique angle.

In one embodiment, the bottom of the well has a smaller radius than thetop of the well. When the objective lens of the imaging system isfocused on the membrane, the top of the well is out of focus and cancause a bright blur that encroaches on the membrane imaging. This blurcan obscure particles that are near the well wall. Therefore, byincreasing the radius of the top of the well, the most out of focuselements are brought away from the membrane, allowing for a better imageof the outside regions of the membrane.

In one embodiment, imaging is performed on bare membranes not attachedto plates, which provides superior imaging but does not allow an easyway to deliver controlled aliquots of liquid. In one embodiment, theheight of the wells is reduces, or the wells are otherwise defined usinga very thin hydrophobic material that causes water droplets to bubble upand maintain their position. This thin material will in certainembodiments be superior for imaging.

The membranes can vary as well. In one embodiment, the membranes areblack track etched membranes, e.g. such as those commercially availablethrough EDM Millipore (Product name: Isopore black membranes. Productnumber HTBP04700). In one embodiment, the surface texture of themembranes is smooth. It is believed that the black material absorbslight that other membranes would scatter, and preferable membranesappears to have low surface roughness which also helps reduce backgroundscatter. In one embodiment, the membrane plate is placed above a dark,non-reflective material such as felt which helps prevent light fromreflecting back up through the membrane. Felt, which has a texture andcan create bumps on the membrane if placed into direct contact, isseparated from the membranes using a spacer/lip. In one embodiment, thetrack etched membranes can be modified as well, for example by coatingthem with a thin metallic layer such as gold, chromium or aluminum.

With dark field imaging, the membrane roughness scatters a lot andcreates a very high intensity background, resulting in lower SNR. Theinstant method is superior to dark field illumination, and one possibleexplanation is that the illumination angle in the instant case is moreoblique. In other words, as the angle of illumination becomes closer toparallel and co-planar with the membrane the better the particles willstand out from the membrane. In certain embodiments, with at least someprotein aggregates, bright field provides a better illumination methodthen oblique angle side illumination. Thus, both methods can be used tocreate a composite image. A ratio of the two imaging techniques canaccording to the various embodiment be a powerful tool to count anddifferentiate types of particles. As shown in FIGS. 9C-9E, one or moreside illumination images 9C can be combined with one or more brightfield illumination images 9D to create a composite image 9E. Theparticles shown are on a membrane and are a mixture of polymericparticles and protein aggregates that have been generated by successiveexpansions and dilations of the liquid-air interface using a tuberotator. The first image shown in FIG. 9C is taken using the obliquelight technique. In this image, the polymeric particle in the bottomright stands out sharply. No other particles are easily distinguished.In the second image shown in FIG. 9D, bright field illumination using a455 nm center wavelength (blue) LED is applied to the particles. Brightfield illumination in this example consists of light that is shined fromabove the sample through the objective lens. In FIG. 9D both the proteinaggregates and the polymeric particle are visible. Though most particlesare more easily distinguished from the background using sideillumination, it is believed that these particular protein aggregatesare more difficult to see due to their flexible and deformable naturewhich causes them to lie flat. FIG. 9E is a composite formed by blendingthe two modes of imaging shown in FIGS. 9C and 9D. The composite imageshows how particles of different materials can appear very differentunder different illumination conditions and that this can be aneffective way to characterize particles. The red particles are proteinaggregates and the yellow one is a polymer. Thus, in one embodiment,particles are characterized by bright filed illumination only. In oneembodiment, particles are characterized by side angle illumination only.In one embodiment, particles are identified by a combination of brightfield and side angle illumination.

Different plate configurations can be utilized according to embodimentsof the invention, and as shown in FIGS. 10A-10D. FIG. 10A shows a plate900 and membrane 902 where the plate 900 is constructed as a clearpolycarbonate plate. FIG. 10B shows a clear polycarbonate plate 910 anda membrane 902. In certain embodiments, the membrane is a polymer ormetallica membrane. The top face of the plate 910 is covered by anopaque material 912. FIG. 10C shows a polycarbonate plate 920 and amembrane 902. The plate 920 has been painted with a reflective coating922. FIG. 10D shows a bright white polycarbonate plate 930 and amembrane 902. The best images came from 1 and 3. The embodiment in FIG.10A has particular advantages because it is believed the light entersthrough both the top face of the plate (and through neighbor well walls)and exits through the well walls of the well to be imaged. FIG. 10C hasparticular advantages because it is believed oblique light enters thewell and is reflected off the walls till it reaches the surface, stilltraveling at an oblique angle.

With reference now to FIGS. 11A-11D, again a key advantage toembodiments described herein is that the illuminating light hits theparticles at a very shallow angle which provides good contrast againstthe background material. The illuminating light in certain embodimentsgoes through plate material before interacting with the sample, thus theplate can be designed with materials and features that help to transmitthe light. Right now, much of the light enters a well through itsneighboring wells. In certain embodiments, optical features areincorporated into the plate material itself, including mirrors, lensesand prisms to transmit, focus and guide the light for improved contrastbetween particles (or other objects) and background and at higher lightintensities (which allows camera exposures to be shorter, leading totime savings and performance improvement). In certain embodiments, thebulk of the plate material or the plate wall is designed to act as alight guide. In the examples shown, an optical feature can beincorporated into the plate material (see FIG. 11A), such as a materialhaving a particular refractive index, a light source can be located inan adjacent well (see FIG. 11B), a surface of the plate material canhave a particular non-linear geometry, such as a Fresnel lens (see FIG.11C), and the plate material can have a lens design (see FIG. 11D).

With reference now to FIG. 12, to create the membrane plate, a thermalbonding technique can be utilized according to one embodiment. The32-well plate 1006 and the membrane 1008 are placed into a compressionjig having a top 1002 bottom 1004 and the jig is placed into a heatedpress. Heat is only applied to the side of the press where bonding takesplace. Then, pressure is applied to carry out the bonding. Both themembrane and the plate itself are made of polycarbonate. Polycarbonateplates were chosen so that they match the material of the membrane sothat thermal bonding could take place. In order to prevent platedeformation during the hot bonding technique the compression jig top1002 makes contact with the plate 1006 only via some protruding ringswhich localize the heat to the desired bonding region. Furthermore, themembrane 1008 in the center of the wells are kept comparatively coolthrough the use of pins 1010 on the compression jig bottom 1004. Thesepins 1010 are in close proximity to the membranes 108 during the heatingprocess and will cause a local temperature decrease. In certainembodiments, the 32-well plate has a protruding “bead” material aroundeach well that acts as the melting region. In certain embodiments, thebead has a triangular profile such that the tip of the triangle is incontact with the membrane and melts first. This method makes bondingmore robust, controlled and strong.

With reference now to FIGS. 13A and 13B, in one embodiment, a wickingmaterial can be utilized in a vacuum manifold. In order to capture aprecise image of captured particles, it is important for the membrane tobe dry and free of sample droplets that may remain due to surfacetension. The vacuum manifold that is used to pull sample through thepores of the membrane can be modified by adding a wicking material (seeFIG. 13A) that contacts the bottom surface of the plate and membrane.The wicking material draws bulk liquid droplets away from the plate viaosmotic pressure (see FIG. 13B). After the sample has been completelyprocessed through the membrane, the plate is lifted off of the vacuummanifold and away from the wicking material. Any remaining liquid in themicroscopic membrane pores rapidly evaporates (<1 s) and the plate isimmediately ready for imaging.

Tracer particles can be utilized to improve image processing. Due tovariations between different membranes, different wells in the plate,different lighting conditions, due to camera noise and a variety ofother sources, comparing images quantitatively is always difficult.Tracer particles of a known shape and size can be added to each well.These tracer particles assist with image balancing, backgroundsubtraction and thresholding calculations because. In one embodiment,polystyrene and polycaprolactone of 10 and 15 microns in size isutilized respectively. In one embodiment, a heat or chemical treatmentis used to fix the particles onto the surface so that they do not moveduring sample loading. For example, polycaprolactone can be heated at 65C for 1 minute. Polystyrene can be treated with nitric acid or heat upto 150 C (so as to prevent membrane deformation). In certainembodiments, innate features on the membranes such as the themselves andthe background texture are utilized to improve processing.

In certain embodiments, illumination with the highest signal to noiseratio of particle to background occurs was when there is no materialaround the membrane (e.g. if the well plate is removed from the membraneand there are no walls for the light to travel through). In oneembodiment, either the well plate itself is removed from the membranefor imaging, or alternatively a hydrophobic material is patterned in ashape that leaves wells behind. Thus, in these embodiments, the wallsare not physical but chemical.

Advantageously, systems and methods according to embodiments describedherein can process a range of low to high volume (e.g. ranges of 10 μL-1mL or 50 μL-1 mL) at speeds of <1 min per well. This low volumesubvisible particle analysis (e.g. at least down to 10 μL or 50 μL) isless than 2-10× of state of the art. The premier sub-visible particleanalysis techniques are FLOWCAM and MFI. Because they analyze particlein a fluidic stream, they require lots of dead volume and tubing.

Low volume processing is important. During formulation selection(researchers have already found an API candidate and now it's time toput it in a stable form that can be intaken by the body), researchers donot have a lot of material to play with, unlike in manufacturing (once aformulation has been selected and is being scaled up). Many times, theyjust have 1 mL per sample for the entire initiative. They would like tobe able to do sub-visible particle analysis to test several formulationsbut currently cannot given tool volume requirements. The approachesdescribed herein can do low and high volume because instead of requiringlarge fluidic reservoir and lots of tubing (dead volume), researchersjust manually pipette samples into the wells, like in a standard plate.The instant embodiments can do large volumes as well, with thelimitation not being volume but particle concentration, if too manyparticles filter is clogged, affecting both imaging and flow.

The embodiments described herein also operate at a higher speed orthroughput. In traditional flow imaging (both FLOWCAM and MFI),particles flow in a fluid channel/flow cell and images are taken as eachparticle flows by. They take multiple images per second to captureindividual particles as they flow by. To analyze a 1 ml sample takes10-15 minutes/sample and the produced files are enormous (tons ofimages). The instant embodiments are faster due to the flow and imagingconditions. Regarding flow, samples are loaded independently in eachwell (pipetted in). Then the sample is flown through the mesh/sieve at ahigh flow rate after applying vacuum. Particles larger than the meshpore size are stuck while the rest of the fluid flows through. Thisprocess can take less than a second. The flow rate here is 100-1000×faster than MFI and FLOWCAM. This increases throughput from analyzing 1ml in 10-15 mins to 1 ml in a few seconds. In addition, the analysis ofthe embodiments shows that the shear rate is low (embodiments do notaffect the proteins as they are clogged on the filter). Regardingimaging, the instant embodiments then take a wide field image of theentire mesh which now has particles spread all over. Thus, instead oftaking individual pictures hoping that a particle is there (see MFIfigure above) instant embodiments flow all the particles quickly andonce they are stuck we can take a single image that captures all theparticles. Instant embodiments take a few pictures, which contain all ofthe particles, and can do this under different illumination conditionsto extract different physical and chemical parameters from theparticles.

Sample recovery is another important aspect of the disclosedembodiments. Unlike traditional flow imaging methods or most particleanalysis methods, the consumables of the instant embodiments trap theparticles (they don't flush down the sink or to disposal) and thereforecan be recovered for additional analysis. This includes additionalassays (stability assays, activity assays, solubility assays), or usingother inspection technologies including mass spec, FTIR, UV-VIS, SEM,EDX, Raman, etc. Sample banking is important since allows researchers togo back and retrace their processes.

Embodiments described herein are applicable to a number of industries.For example, identifying or characterizing particles inbiopharmaceutical samples, including protein aggregates, excipients,silicone oil droplets, air bubbles, glass, metals, fibrous materials,and other intrinsic and extrinsic particles encountered in theformulation and manufacturing processes of the biopharmaceuticalformulation process; particles in aquatic research, including freshwater, wastewater and marine research including algae; particles in theoil and gas industry including drilling fluid, frac materials and fuels;and particles in the paints and cements industry.

With reference now to FIG. 14A, in one embodiment, a method 1100 forcharacterizing at least one particle from a fluid sample includes thesteps of introducing a fluid sample onto a filter 1101, illuminating theat least one particle at an oblique angle 1102, and imaging theilluminated at least one particle as it rests on the filter forcharacterizing the at least one particle 1103. In one embodiment, themethod includes the steps of illuminating the at least one particleusing bright field illumination. In one embodiment, the method includesthe steps of generating a composite image based on the oblique angle andbright field illumination. In one embodiment, the method includes thesteps of illuminating the at least one particle by radially surroundingthe at least one particle with a plurality of illuminating devices andilluminating the at least one particle from an oblique angle. In oneembodiment, the method includes the steps of illuminating the at leastone particle in one of a plurality of wells disposed on a well plate. Inone embodiment, the method includes the steps of charactering the atleast one particle based on imaging prior to an after a fluid sample isintroduced onto the filter. In one embodiment, the method includes thesteps of individually and separately imaging each of the plurality ofwells. In one embodiment, the step if imaging comprises imaging at aplurality of heights above the filter. In one embodiment, the pluralityof images comprises a high dynamic range set of exposures. In oneembodiment, the method includes the steps of merging the plurality ofimages together into a single image. In one embodiment, the plurality ofimages comprises replicates. In one embodiment, the method includes thesteps of mathematically registering two or more images. In oneembodiment, the method includes the steps of mathematically registeringbefore and after images. In one embodiment, the method includes thesteps of introducing the fluid sample into a well on a well plate. Inone embodiment, the method includes the steps of illuminating the atleast one particle through an optical feature of the well platecomprising at least one of a mirror, lens and prism. In one embodiment,the method includes the steps of characterizing and identifying amaterial type of the at least one particle using a machine learningalgorithm. In one embodiment, the machine learning algorithm usesobserved features including at least one of size, shape, texture,dark-field intensity and intrinsic fluorescence of particles to buildmodels.

With reference now to FIG. 14B, in one embodiment, a method 1200 forcharacterizing at least one particle from a fluid sample includes thesteps of introducing a fluid sample onto a filter 1201, illuminating theat least one particle using bright field illumination 1202, and imagingthe illuminated at least one particle as it rests on the filter forcharacterizing the at least one particle 1203. In one embodiment, themethod includes the steps of illuminating the at least one particle atan oblique angle or an angle coplanar with a flat plane of the filter.In one embodiment, the method includes the steps of illuminating the atleast one particle by radially surrounding the at least one particlewith a plurality of illuminating devices and illuminating the at leastone particle from an oblique or coplanar angle. In one embodiment, themethod includes the steps of illuminating the at least one particle inone of a plurality of wells disposed on a well plate, wherein each ofthe wells terminates in a filter.

EXPERIMENTAL EXAMPLES

The invention is now described with reference to the following Examples.These Examples are provided for the purpose of illustration only and theinvention should in no way be construed as being limited to theseExamples, but rather should be construed to encompass any and allvariations which become evident as a result of the teaching providedherein.

Without further description, it is believed that one of ordinary skillin the art can, using the preceding description and the followingillustrative examples, make and utilize the present invention andpractice the claimed methods. The following working examples therefore,specifically point out the preferred embodiments of the presentinvention, and are not to be construed as limiting in any way theremainder of the disclosure.

In certain embodiments, 3D printed flow cells are used to hold andperform experiments on the electroformed micro-meshes in isolation(without microflow imaging). In other embodiments, a flow cell isdesigned to allow for simultaneous microflow imaging and delivery ofsamples through our filter chip. In certain embodiments, the designmust: (1) contain a fluid channel of specific thickness such that theentire height of the microchannel lays within the depth of field of ourimaging system while allowing the desired particle size range to passthrough freely, (2) the assembly of the chip and flow cell should besimple and robust (e.g. free of leaks or introduction of contaminants),and (3) the flow cell thickness above the surface of the micromesh mustbe sufficiently thin to allow access for and imaging by a highmagnification objective for spectroscopic characterization/imaging. Inone embodiment, die-cut silicone tapes with 50 and 100 μm thicknesses asgaskets between the microfabricated chip and a CaF₂ window for IRtransparency are used. Custom stainless steel holders are machined witho-rings and tube fittings to perform micro- to macrofluidic coupling, ina fashion similar to the NanoTweezer Surface product. The machinedholder will also house illumination optics for microflow imaging.Alternatively, a micro-gasket can be employed instead of a siliconetape, allowing for removal of an ordinary piece of glass prior tospectroscopy. In other embodiments, the flow cell is not designed forimaging, but instead the particles will be imaged on the micromesh. Inthis embodiment, the flow cell does not need to be transparent. Sinceimaging will take place on the micromesh, the thickness of the flow cellabove the micromesh as well as the flow cell cover need to becontrolled. Ideally a quartz cover slip can be used with as thin a gapas possible that will allow the particles to flow through. However, aglass cover slip can be used as well. The cover slips may also beremoved before spectroscopy in order to get the best imaging andspectroscopy results.

In certain embodiments, wafers will consist of 2, 3, and 5 um circularpores designs. Samples are prepared to contain approximately 2,000particles, 5-50 microns in diameter, per mL, such that the resistance ofeach chip due to pore clogging changes less than 5% for each pore size.Each sample (1 mL volume) is passed through the mesh at a controlledflow rate ranging from 0.25-5 mL/min (comparable to current flow ratesemployed in microflow imaging). Flow rates are maintained via an activefeedback loop using an in-line flow sensor and our pressure regulationmodule. Capture efficiencies for each pore size are quantified byperforming particle counts via microscopy on each mesh, and comparingcounts to the initial concentration as determined by microflow imaging.Second, the effect of particle loading on capture efficiency and flowrate is determined, and hence the capacity of each filter design. Highconcentration samples (10,000 particles/mL) are flowed through each chipwhile monitoring the volumetric flow rate at a constant appliedpressure. As particles are trapped on the mesh, the flow rate isexpected to decrease until the device clogs (flow rate=0) or captureefficiency drops due to particles being pushed through the pores, andflow rate stabilizes at a low value. Through this experiment it isexpected to get both an estimate of the filter capacity, in terms oftotal number of trapped particles, as well as an understanding of theflow rate (at constant pressure) signatures as the filter capacity isreached.

Preliminary data has been acquired using a prototype apparatusconsisting of an Olympus BX-51 microscope configured for bright fieldillumination, a spectrometer (Tornado) and a 785 nm laser (InnovativePhotonics Solutions) for Raman analysis, and a CMOS camera (Basler).Incorporated into the setup is a focal plan array detector forhigh-throughput compositional mapping. With a Raman system, spectra isacquired at a rate of one every two seconds. For a mesh with 5000particles trapped, it would require approximately 3 hours to acquirespectra from every particle, followed by spectral processing andidentification. FTIR imaging utilizing a focal plane array detectorallows simultaneous acquisition of IR absorbance spectra for each pixelof the image. In this format, extremely high volumes of spectra can beacquired in a short time. For example, utilizing a focal plane arrayfrom Bruker, a detector we have arranged to test in Phase II, highresolution spectral imaging is obtained at about 0.5 mm²/min (16 scanaverage, 4 cm⁻¹ resolution, 5.5 um pixel resolution), which translatesto approximately 10 minutes to acquire spectra (147,456 spectra total)for the entire filter area. We evaluate both the throughput and statedresolution of two FPA detector providers with protein aggregate samplestrapped on microfabricated chips to determine the best components basedon performance and cost of integration.

A custom microflow imaging system is optimized for chip design andchannel dimensions. A custom board housing a high power 460 nm LED(Luminus Devices Inc.) and heat sink is fixed to the bottom of the flowassembly. Control of the LED is integrated with a CMOS microscope camera(Basler Inc.) to flash during each image acquisition at a rate of 62fps, which as described below will be sufficient to sample each particlemultiple times. Prototype software controls both the illuminationintensity and the camera, allowing users to adjust acquisition time,exposure and frequency. Using a 4× objective and a camera sensor size of22.5×16.9 mm, both the mesh and microflow imaging region of the chipwill be visible within the same field of view. Data obtained with ourmicroflow imaging module is validated by comparing with data collectedon commercial microflow imaging systems using spiked samples of particlestandards.

The FTIR spectroscopy platform and the flow imaging components areintegrated into a stand-alone instrument. For hardware integration, aninstrument casing houses a custom, horizontal optical train consistingof a microscope objective (potentially two depending on the requirementsof IR imaging and microflow imaging), a splitter leading to a camera andto the FPA detector. The instrument box also houses the modulatedexcitation source for IR imaging, a pressure control board, and themicroflow imaging unit, consisting of the flow cell and strobed blue LEDillumination.

A software platform implements routine sample analysis. The existingparticle tracking algorithms link particles imaged in the microflowimaging region of the chip to specific locations on our patternedmicromesh. Also automated is the control of both the microflow imagingconditions, flow conditions (pressure and flow rate) and spectralacquisitions, which will serve to improve usability as well asconsistency of results.

In certain embodiments, particle tracking software is used to track andmeasure particles during flow, as well as identification of particlestrapped on the mesh. In certain embodiments, linking of microflowimaging data to the spectra obtained for each particle on the micromeshis implemented. With a typical particle count of around 5000 particlesper 10 mL sample, the fluid volume in our imaging region (˜0.44 μL)contains approximately 0.22 particles at any given instant. At a typicalflow rate of 0.1 mL/min, the fluid volume will be refreshedapproximately 4 times per second. Imaging is implemented at a frame rateof at least a factor of 5× faster than the fluid volume refresh rate toensure a particle is imaged several times between entering the imagingregion and being captured on the mesh. The implementation of trackingwill enhance our sampling rate over conventional micro-flow imaging,which intentionally incorporates gaps between sampling to ensureparticles are not counted twice.

In certain embodiments, the addition of particle tracking to imageprocessing will build largely on the NanoTweezer Surface platform, whichperforms spot detection, linking of spots, and reporting of particlecharacteristics and tracking characteristics on 80,000 frame datasets(which would correspond to ˜45 minutes of microflow imaging data at 30fps) containing 10-20 particles/frame in less than 10 minutes. Inaddition, a software package that utilizes graphics processing units(GPUs) can be implemented to reduce our processing time by a factor of10.

We first attempt to track particles from entry into the channel untilthey are trapped on the mesh. Once a particle leaves the microflowimaging region, it may be difficult to track the particle above thepatterned background of the micromesh. When a particle is trapped by apore, however, a distinct reduction in transmitted light is observed atthe pore location. If we are unable to track particles above the mesh,we can use this feature to link trapped particle locations to theirmicroflow images. The low concentration of particles in a typicalBiopharma samples suggests that on average a new particle will beobserved every 1-10 frames (for concentrations ranging from 100-1000particles/mL). We expect this rarity will facilitate linking particlesobserved in the microflow section to newly trapped particles on the meshusing local changes in transmitted light. As an additional riskmitigation step, we will evaluate whether there is useful correlationbetween observed streamline locations of flowing particles and trappedlocations on the mesh, under the laminar flow conditions found in ourmicrochannel.

In certain embodiments, imaging conditions, spectral acquisitionconditions, and flow conditions are determined by the user, and manuallyset and executed using software packages developed in-house.Furthermore, spectral signatures/peak locations of suspected materialsare manually added to our program in order to generate compositionalmaps. In addition to eliminating the need for intensive training,automation via software is used to improve run to run consistency.Automation of key steps in setting up and running a successfulexperiment is preferable. First, exposure time and illuminationfrequency and intensity is automatically adjusted to maximize imagequality for particles introduced by the sample. As discussed earlier,flow parameters will be monitored such that the run is automaticallyterminated when a predetermined volume has been processed, or when adegree of particle load on the filter is achieved. When spectralacquisition is executed, the mesh will be automatically sampled over itsentire area. Acquired spectra will be matched against a spectraldatabase to identify composition.

In one embodiment, a database utilizing p fluorescence imaging isutilized. Different particles can have different scattering propertiesat different excitation wavelengths. A model is created or trained usingmachine learning. In certain embodiments, fluorescence is used in placeof raman/FTIR because it is high signal. Flourescence is not as specificas the spectral signatures achieved with raman/FTIR, but because thetypes of contaminants are limited (protein aggregates, metals, rubber,etc), the full power of complex spectroscopy is not necessary. Incertain embodiments, 3 channels of fluorescence and a good model foreach particle type is sufficient for particle characterization andidentification.

One method to obtain rapid particle composition information in the sizerange that is most important for certain embodiments of the device isthrough spectroscopy. Of the various spectroscopic methods available,Mass Spectrometry is typically used more upstream during drug design andsuffers from very complex sample prep. Optical spectroscopy, like FTIRis more commonly used to ensure that the bonds of interest are present.FTIR is fairly high throughput compared for example to Raman. Ramanspectroscopy while potentially more chemically sensitive suffers fromseveral orders of magnitude lower signal and throughput. Furthermore,neither FTIR nor Raman are used in routine biopharm analysis, and onlyin forensic analysis (once things have gone wrong) due to their lowthroughput. Routinely identifying protein aggregates is key with thisresearch tool. With the protein aggregate information, researchers canattest whether they should relook at their formulation or whether otherparticles may be present. In order to achieve this measurement on aroutine and not low throughput basis, multichannel fluorescence isutilized and prioritized over other technologies like FTIR or Raman.Fluorescence can accomplish the “crude spectroscopy” with the throughputneeded to meet the routine requirement. In one embodiment, multi-channelfluorescence is utilized, since particles will exhibit differentfluorescent properties at different excitation wavelengths. For example,using R,G,B, each channel may exhibit unique light scatteringcharacteristics of the particle. In one embodiment, the fluorescenceimaging uses intrinsic multi-channel based fluorescence. In oneembodiment, the fluorescence imaging uses labeled multichannel basedfluorescence.

Fluorescent imaging is a sensitive and non-invasive method forquantitatively evaluating protein autofluorescence (Ghisaidoobe andChung, Int. J. Mol. Sci., 2014, 15, 22518). Protein and proteinaggregates have an intrinsic fluorescence in the deep-UV wavelengthrange (excitation ˜280 nm, emission 303 nm and 350 nm) due to thepresence of constituent aromatic amino acids. In addition, proteins thathave aggregated together deliver additional fluorescent energies in thelarger UV-blue wavelengths (excitation ˜340 nm, emission 425, 445, 470,and 500 nm) (Chan, Kaminski, Schierle, Kumita et al, Analyst, 2013, 138,2156) & (Shukla, Mukherjee, Sharma et al, Archives of Biochemistry andBiophysics 2004, 428, 144). It is speculated that this aggregatefluorescence stems from the presence of higher-order aggregate structureor resonant bond features. Additionally, it is thought that degree ofaggregation as well as more specific aggregate structure or type (forexample random vs. fibril) can be weaned by evaluation of theseaggregate-specific wavelengths. Concomitantly, additional particulatematerial types that are inherent to the biopharma manufacturing streamand processing equipment, may end up aggregating along with protein orconstitute sample impurity. These materials, potentially includingpolymers, glasses, and metals, can be evaluated rapidly and routinelyusing fluorescent spectroscopy.

In addition to non-invasive evaluation of autofluorescence using opticalspectroscopy, a system utilizing a chip with a geometric grid, such as ahex-shaped grid as described herein, will have an optional abilityallowing users to actively label particulate material using afluorescent probe or dye. The system can be simply adapted toeffectively flow a fluorescent dye or antibody solution over capturedmaterial on the hex-shaped grid, followed by a rinse of non-boundmaterial. The use of protein staining can allow for enhanced and veryspecific protein aggregate signal or other material types. For example,Thioflavin-T has been shown to associate rapidly with multimeric β-sheetcontaining specific amyloid fibrils (Groenning, Olsen, van de Weert etal 07'). With this added capacity, users can take advantage ofalternative specific and powerful fluorescent chemical/antibodyconjugation methods that have a long history of validation and value inindustrial applications, to optionally follow up on their opticalspectroscopy results with deeper more specific evaluations of capturedparticulate material, should the need arise. As described herein,combining bright field with side illumination yields excellentdifferentiation between particles of different materials.

In certain embodiments, there are three stages to the techniques of theembodiments disclosed herein: microflow imaging (MFI), filtration andspectroscopic identification. Regarding MFI, high quality particleimaging in a flow cell using a MFI setup has been demonstrated (seeFIGS. 15A-15C). A simulated biopharma sample was prepared by aheat-treatment method of generating protein aggregates followed byspiking with additional polymeric microparticles (PMMA and PCL). Thesample was then run on an MFI system. A gallery of particles (FIG. 15A),post-processed particle tracking (FIG. 15B) and a histogram of particlesizes (FIG. 15C) is shown.

With reference now to FIGS. 16A and 16B, a method to capture a simulatedprotein therapeutic sample consisting of a mixture of protein aggregatesand particles in a high protein content solution on a microfabricatedgrid was developed, and trapping subvisible particles with electroformedmeshes is demonstrated. Electroformed meshes were die cut into 20 mm²circles and loaded into a custom built flow cell. Samples were added toa well and pulled through the mesh using vacuum pressure regulated by apressure controller as illustrated in the diagram of FIG. 16A. Followingseparations, samples were then imaged under a 4× objective. The image ofFIG. 16B shows a mesh following separation of a mixture of aggregatedlysozyme (2-50 um), stainless steel beads (10-20 um), and latex beads(15 um). Regarding identification, following the filtration step, thecaptured particles were scanned using a Raman microscope with roboticstage (see FIG. 17A) and with an FTIR microscope (see FIG. 17B).Microscope images and spectroscopy scans of particles caught on amicrofabricated gold mesh with 17 micron pores are shown. A mixture ofprotein aggregates and polymeric particles were collected with thefilters and scanned. Raman data Red outlines were added to showtopographical similarities—they do not outline individual particleswhich are much smaller) (see FIG. 17A). Additional unknown contaminantswere identified and shown on the legend. Rapid FTIR imaging of a samplewith a heat map of the Amid I peak at 1700 wavenumber which identifiesproteinaceous material (see FIG. 17B).

MFI is a quantitative particle imaging technology that has proliferatedin the protein therapeutics space (and others) for routine measurements(˜10 per day) of protein therapeutic samples. Whereas traditional MFIsystems only take one image of a particle, embodiment of the inventionuses a higher frame rate in order to track particles to their landingsite on the grid. We have implemented an MFI setup in order to obtainhigh fidelity particle images. In FIG. 12, we demonstrate our ability tocapture images of particles in a flow cell, carry out particle trackingand analyze the particles to provide an estimated diameter.Pharmaceutical scientists often look at these particle galleries (whichare displayed on their MFI systems) and wonder what these particles arethat contaminate their otherwise pristine formulations.

In order to acquire certain preliminary data, we utilize the NanoTweezerplatform (launched via a NSF SBIR). The NanoTweezer system also uses aflow cell, particle imaging and particle tracking. Live particletracking is implemented via GPU processing, which will also vastlyimprove the processing time for MFI. Lastly, since MFI requires carefulflow rate control, the same pneumatically controlled pumping system isutilized that is used on NanoTwezer. This uses an inline flow sensor aspart of a PID feedback loop to obtain accurate flow rate (within 5% ofrequested flow rate) with good stability (<1% c.v.)

This proof-of-concept data shows the ability to design microfabricatedstructures and use them to capture and subsequently identify relevantmaterials in a protein therapeutic setting. Several differentelectroformed micropore grids were used with pore sizes ranging from 11μm to 25 μm and using both nickel and gold as the material. The bestresults when combined with Raman spectroscopy was gold, which hasexcellent properties for use with Raman spectroscopy—low background,high reflectance and sometimes even enhancement via surface enhancedRaman spectroscopy (not used here). The same simulated proteintherapeutic sample as described in the previous section was used andcaptured particulates on the pores. Due to the exceedingly high openarea of these filters, an entire mL of sample containing vastly moreparticles than a real sample was filtered within seconds. Only about 100millibars of vacuum pressure were needed. FIG. 13 shows the results witha 17 μm pore gold mesh.

Accordingly, imaging, trapping and chemical identification of proteinaggregates mixed with standard particle samples on prototype meshdevices has been demonstrated using electroformed meshes integrated intoflow cells, a custom-built Raman microscope, and prototype software todrive each component. In certain embodiments, instead of usingelectro-formed meshes, a microfabricated meshes built on solid supportswas used, enabling scalable integration into a flow cell. In certainembodiments, software and hardware is integrated into a single analysisinstrument. In certain embodiments, methods are implemented in softwarefor automated mapping of particles imaged during flow to locations onthe mesh. The operation and data analysis steps performed by the deviceare automated.

For certain embodiments utilizing a well plate, image acquisition wasachieved using high dynamic range imaging. HDRI is one photographictechnique that creates a composite image created from capturing imagesfrom low to high exposures (in our case we take 4 to 10 images togenerate a single composite HDR image). This allows one to capture awide range of particle sizes without losing data due to overexposure.The camera that captures the images is a wide field camera in order tobe able to take one or a few HDR pictures per well. If multiple sets ofimages are required to capture the entirety of the membrane, they can bestitched together.

As explained above, the membranes we are using for filter material werenot perfectly devoid of background features; they have a texture, theremay be some contamination in the form of particles, or even scratches inthe material itself. In order to accurately count the number ofparticles added to the membrane surface it is necessary to account forthis background. In one embodiment, we do this by taking a “before”image, and an “after” sample image, then compare the two using softwarealgorithms. One method is as follows: Step 1: Image registration. Thebefore and after images are registered so that features shared by thetwo images are aligned in x, y and rotation as well as introducing ascaling factor to one of the images and allowing for some deformation ofthe image so that features in the before and after image are at the samex and y position. Step 2: Remove light ring. The images are rectangularviews of a circular object. The extra areas of the image outside of thecircular membrane are removed. Step 3: SNR Transformation. This levelsthe images to account for uneven lighting on the image. There are manyways to do this. Including rolling ball method, high pass filters, Fastfourier transforms, etc. Images are equalized by normalizing intensitiesto the local median value. This is accomplished by taking the higherintensity image and dividing the intensity of each pixel by the factorF, where F=median of a large neighborhood around the pixel. This is doneto both the “before” and “after” images Step 4: Background subtract. Inorder to identify particles with respect to the membrane texture thetransformed images are divided be each other pixel-wise. Thus we obtaintwo ratio images “before”/“after” and “after”/“before”, the second isused to detect particles that got trapped after the “before” image wastaken Step 5: Threshold the “before” image. We use a thresholdingtechnique to identify particles. We do this by thresholding an imagesuch that all pixels that are brighter than the threshold bright pixelsturn white (value 1) and all pixels dimmer than the threshold turn black(value 0). Several threshold values can be used, but we typically usesome multiple of the average background intensity. Currently we use 2times the background intensity. After this some additional processingcan be done to close edges and fill gaps. A “blob” of connected whitepixels is considered a particle. See FIG. 18A “Before Image”. Step 6.The mask is applied to the image so that the locations of particles canbe measured to provide brightness, morphological and positional dataabout each particle. These details are recorded in a table. Step 7: Inthe “after”/“before” image we can pick a threshold higher than 1 todetect the added particles. To pick the optimal threshold the followingsteps are performed according to certain embodiments. Method 1: We pickan “after”/“before” image threshold by the following method. We makeseveral guesses at a threshold value. For each guess, we apply thethreshold to the image, identify particles and compile a list ofparticles found using this threshold. Then we compare the afterparticles to the before particles. For each “before” particle we findthe closest “after” particle and assign the connected pair a “benefit”value. The lower the distance of the particles' centroids, the moresimilar the areas of the particles, the more similar the brightness, themore similar the morphological features are, the higher the benefit. Notall of these attributes need to be used though. For the guessed “after”image threshold, a total value for connected particle “benefit” iscalculated by summing up each connected particle pair. Next, we attemptto maximize the benefit by picking the best “after” threshold. This canbe done by iteratively making guesses and changing the threshold aftereach guess in the direction that appears to be correct. This is a kindof feedback loop. Alternatively, a series of predetermined thresholdscan be calculated and then a curve fitting or estimation method to canbe used pick the threshold with the maximum “benefit”. Ideally, everyparticle found in the before image should correspond to a particle inthe after image. However, some particles may have shifted ordisappeared. See FIGS. 18A-18C. The before image (FIG. 18A) wasthresholded and 5 particles were identified in this section of thebefore image. After identifying a good “after” threshold as described instep 7 above, the after image (FIG. 18B) identified 16 particles. The 5particles from the before image were connected to the after image andare shown with arrows in FIG. 18C. There will therefore be 11 newparticles reported in this section of the membrane. Method 2: For eachparticle found in the before image, we look at the same region in theafter image and determine what threshold would produce the highest“benefit”. After doing this for each particle we pick a threshold, oreven multiple thresholds that give the best result. Step 8: Attempt tofind particles that have shifted. Some particles that were in the beforeimage but not the after image can attempt to be located by looking forparticles in the after image that have the same morphological andbrightness characteristics. Step 9: After the connected particles havebeen identified, the remainder of the identified particles in the afterimage can be reported.

In one aspect, different particles can be distinguished based ondifferences in intrinsic fluorescence properties. Fluorescent imaging isa reliable and rapid method for quantitatively evaluating proteinauto-fluorescence, since they fluorescence in the deep-UV wavelengthrange (excitation ˜280 nm, emission 303 nm and 350 nm) due to thepresence of constituent aromatic amino acids. To test fluorescencecharacterization, we prepared a mock biopharmaceutical sample consistingof protein aggregates (bovine granulocyte colony stimulating factor) anda polydisperse mixture of polystyrene particles. In addition tocapturing a ‘dark-field’ image used for counting, fluorescence typingimages were acquired as well using a protein intrinsic fluorescencechannel. The results, shown in FIGS. 19A and 19B show easydifferentiation between the two types of materials. Different imagingmodes shown in the top of the plot can be combined and processed to“clip out” individual particles and plot their characteristics. Ingeneral, particles with the least amount of protein signal (the polymermicrospheres) match the morphology of the spiked non-protein particles.Rounder particles (polystyrene) were also the ones that exhibited theleast fluorescence in the protein channel. In addition, proteinaggregates fluoresce in the larger UV-blue wavelengths (excitation ˜340nm, emission 425, 445, 470, and 500 nm), with this aggregatefluorescence stems from the presence of higher-order aggregate structureor resonant bond features, and may even reveal information of theaggregate structure (e.g. random vs. fibril) and to distinguish betweendissolvable (dangerous) vs. non-dissolvable aggregates. The system canalso distinguish between ETFE polymeric standards that mimic proteinaggregates with actual protein aggregates, as shown in FIG. 19B.

In one embodiment, the imaging device is configured to image the filterboth before and after the at least one particle is captured on thefilter. In one embodiment, the before and after images are processedtogether using algorithms that processes them to find differences. Inone embodiment, the imaging device is configured to take a plurality ofimages at a plurality of heights above the filter. In one embodiment,the plurality of images consists of a high dynamic range set ofexposures. In one embodiment, the plurality of images are mergedtogether into a single image. In one embodiment, the plurality of imagescomprises replicates. In one embodiment, the plurality of images atdifferent heights are merged together into a single image. In oneembodiment, two or more images are mathematically registered. In oneembodiment, the before and after images are mathematically registered.In one embodiment, a median filter is used to process the image.

Machine learning algorithms were used to distinguish between differentparticle types. Different features were used, including size, shape andintrinsic fluorescence, to build the models. To test feasibility ofusing machine learning to classify particles, we collected a data setcontaining ˜40,000 particle images. The training set consisted ofprotein aggregates, glass particles, cellulose particles, stainlesssteel particles, ETFE standards from NIST, and latex beads. 50% of thedataset was set aside to be used as a testing set for final evaluationof the predictive model. The remaining 50% was used to train a machinelearning algorithm, xgboost, short for extreme gradient boosting. Forthis model, only the intensity signal-to-noise ratios for a darkfieldillumination channel and each fluorescence channel were used asfeatures. The building of the model is shown schematically in FIG. 20A.In short, images collected of known particles are analyzed and theextracted parameters are used to construct a series of decision trees,which are then used to classify particles of unknown type. First, anxgboost model was trained with the training data to perform predictionson the simple case of proteinaceous (protein aggregates) vsnon-proteinaceous (all other particle types) typing. The model consistedof 100 decision trees, and yielded an accuracy on the testing set of95.5%, sensitivity of 97.9%, and specificity of 77.4%. In this case,16,238 non-protein particles were correctly identified as beingnon-protein, and 342 particles were mistakenly classified as proteins.1727 protein aggregates were correctly identified as proteinaceousparticles, and 504 were incorrectly classified as non-proteinaceous. Anexample output plot of protein vs non-protein identification is shown inFIG. 20B, collected on a mixed sample containing protein aggregates andlatex beads. For multi-class classification, the same xgboost algorithmwas trained to identify individual particle types, instead of simpleproteinaceous or non-proteinaceous determination. The same training datawas used (SNR for each channel), and an xgboost model consisting of 100decision trees yielded a typing accuracy of 88.6%, with specificitiesranging from 89.7-99.7%, and sensitivities ranging from 69.2-95.1%. Theconfusion matrix of labeled versus predictive cases is shown in FIG.19B, where the diagonal represents correctly identified cases.

With reference now to FIGS. 21A and 21B, the membrane where particlesare captured and imaged were usually never perfectly in focus throughoutthe whole image (FIG. 21A) for one image. There may be local membranedeformations, spherical aberration at the image edges, the presence oflarge particles which are partially out of focus, and mechanicalmisalignment or flatness. We employ a technique known as focus fusion tocombat these issues. The technique is not novel, but, when used inconjunction with all of the other techniques, is an essential part ofwhat makes the overall imaging work. Focus fusion works by taking manyimages at different heights above the substrate and merging them into asingle “focus fused” image (FIG. 21B). This dramatically improves thefeature detail across the whole image, enabling the system to accuratelyremove background texture, acquire physical parameters including counts,size and provides higher fidelity regarding intensity measurements, andensures that particles that are out of focus do not appear large anddim. This technique is used in combination with other imaging techniquessuch as HDR imaging mentioned earlier and taking multiple averages ofthe same image to improve image quality and reduce noise.

The disclosures of each and every patent, patent application, andpublication cited herein are hereby incorporated herein by reference intheir entirety. While this invention has been disclosed with referenceto specific embodiments, it is apparent that other embodiments andvariations of this invention may be devised by others skilled in the artwithout departing from the true spirit and scope of the invention.

1. A system for characterizing at least one particle from a fluid samplecomprising: a filter disposed upstream of an outlet; a luminaireconfigured to illuminate the at least one particle at an oblique angle;and an imaging device configured to capture and process images of theilluminated at least one particle as it rests on the filter forcharacterizing the at least one particle.
 2. The system of claim 1further comprising: a luminaire configured to illuminate the at leastone particle at an angle coplanar with a flat plane of the filter. 3.The system of claim 1, wherein the luminaire comprises a plurality ofilluminating devices.
 4. The system of claim 3, wherein the plurality ofilluminating devices are disposed radially around and directed inwardlytowards the filter.
 5. (canceled)
 6. The system of claim 1 furthercomprising: a luminaire configured to illuminate the at least oneparticle by bright field illumination.
 7. The system of claim 1, whereinthe imaging system is configured to generate a composite imagecomprising oblique angle illumination and bright field illumination. 8.The system of claim 7, wherein the imaging system is configured tocharacterize a plurality of particles based on a combination of obliqueangle illumination and bright field illumination.
 9. The system of claim1 further comprising a well plate having a plurality of wells that eachterminate on a filter.
 10. The system of claim 9, wherein the filter isa membrane.
 11. (canceled)
 12. The system of claim 9, wherein themembrane surface has a low surface roughness.
 13. The system of claim 9,wherein the well plate comprises a transparent material. 14-17.(canceled)
 18. The system of claim 9 further comprising: a vacuummanifold connected to a negative pressure source and configured forfluid communication with the plurality of wells.
 19. The system of claim9, wherein the well plate comprises an optical feature comprising atleast one of a mirror, lens and prism.
 20. The system of claim 1,wherein the imaging device is configured to characterize and identify amaterial type of the at least one particle using a machine learningalgorithm.
 21. The system of claim 20, wherein the machine learningalgorithm uses observed features including at least one of size, shape,texture, dark-field intensity and intrinsic fluorescence of particles tobuild models. 22-24. (canceled)
 25. The system of claim 1, wherein theimaging device is configured to characterize and identify a materialtype of the at least one particle as it rests on the filter usingfluorescent imaging.
 26. The system of claim 25, the fluorescenceimaging comprises intrinsic multi-channel based fluorescence. 27.(canceled)
 28. The system of claim 1, wherein the imaging device isconfigured to image the filter both before and after the at least oneparticle is captured on the filter.
 29. The system of claim 28, whereinthe before and after images are processed together using algorithms thatprocesses them to find differences.
 30. The system of claim 1, whereinthe imaging device is configured to take a plurality of images at aplurality of heights above the filter. 31-73. (canceled)